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The  Increasing  ability  to  tailor  the  designs  of  electronic  circuitry 
to  unique  requirements  and  the  rapid  technological  advances  In  electronics 
tends  to  limit  the  derivation  of  nathematlcal  Cost  Estimating  Relation¬ 
ships  (CER).  Estimates  made  for  electronic  item  costs  therefore  have 
relied  heavily  on  engineering  Judgement  and  analogy  estimating.  This 
report  covers,  In  an  exploratory  sense,  the  weaknesses  of  many  estimates 
by  analogy  and  the  considerations  that  may  be  entertained  to  approve  the 
estimating  procedure. 

It  Is  the  Intent  of  this  report  to  provide  a  critique  of  a  poor  esti¬ 
mate,  Identify  the  basic  problem  areas,  and  provide  suggested  procedures 
to  minimize  the  weaknesses  of  analogy  estimating.  Detailed  step-by-step 
procedures  are  Introduced  and  use  of  some  basic  statistical  measures  are 
presented.  The  report  concludes  with  a  "checklist"  of  questions  the 
estimator  should  ask  himself  to  determine  the  adequacy  of  his  estimating 
endeavor. 
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COST  ESTIMATION  BT  ANALOGY 


I.  Introduction.  Too  often  cost  estimates  based  on  a  comparison  with 
some  other  item  are  made  vhen  a  quick  estimate  is  required  and  there 

are  Insufficient  or  no  historical  data  available  for  a  detailed  estimate. 

The  major  drawback  to  this  estimating  by  analogy  is  that  is  is  essentially 
a  judgment  process  that  requires  considerable  experience  and  expertise. 

Thus,  an  acceptable  analogy  can  not  be  haddy  made  and,  in  fact,  should 
probably  require  more  effort  than  other  methods  such  as  the  use  of  factors 
or  estimating  relationships.  There  are  many  pitfalls  that  must  be  avoided 
In  order  for  the  analogy  method  to  be  successful  and  useful.  In  the  past, 
"engineering  Judgment"  has  been  cited  as  the  rationale  for  this  type  esti¬ 
mate.  While  this  rationale  is  valid,  it  is  too  often  "top-of-the-head, " 
based  on  a  sample  of  1,  not  documented,  and  nonre producible,  resulting  in 
an  estimate  that  cannot  be  evaluated  by  the  user.  Actually,  "engineering 
Judgment"  should  be  representative  of  a  veil  ordered  scientific  reasoning 
process  that  can  be  adequately  defined  and  described  in  detail. 

It  is  the  purpose  of  this  document  to  review  the  common  or  present 
methods  that  appear  prevalent  in  analogy  estimates,  expose  their  short¬ 
comings,  and  establish  procedural  goals  to  consider  in  future  cost  esti¬ 
mating  by  analogy. 

II.  Definition  of  an  Analogy.  Analogy  is  commonly  defined  as  an  agree¬ 
ment  ,  likeness,  or  'Correspondence  between  relations  of  things  to  one 
another;  a  partial  similarity  in  particular  circumstances  on  which  a  com¬ 
parison  may  be  based.  The  comparison  process  is  to  take  note  of  the 
differences  and  similarities  of  the  items  being  compared.  In  cost  esti¬ 
mating,  the  analogy  can  be  based  on  the  likeness  or  correspondence  of  tech¬ 
nical,  physical,  and/or  functional  attributes.  Noting  these  similarities 
to  items  of  known  cost,  a  basic  estimate  can  be  made  and  then  adjusted  to 
provide  for  the  cost  effects  of  the  differences  between  the  items. 

IH.  Customary  Analogy  Estimating. 

A.  Procedures.  One  of  the  methods  of  estimating  by  analogy  which  may 
appear  quite  often  is  a  comparison  of  the  item  to  be  developed  to  an  item 
it  is  replacing.  Ibis  situation  can  be  seen  to  present  itself  frequently 
since  as  basic  hardware  technology  advances,  replacement  equipment  is 
developed  to  expand  performance,  to  reduce  size  and  weight,  or  to  provide 
for  less  expensive  life  cycle  operation.  Many  such  instances  are  of  a  pro¬ 
duct  improvement  or  re-engineering  type  effort.  In  any  event,  such  develop¬ 
ment  efforts  reflect  the  incremental  advances  in  technology  for  a  given 
functional  type  item,  e.g.,  surveillance,  manpack  communication,  etc. 
Approaches  to  estimating  the  costs  per  unit  hardware  these  incremental 
advances  incur,  leads  to  a  common  procedure  that  is  illustrated  in  the 
following  hypothetical  example; 
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Problem:  Estimate  the  unit  cost  of  item  B,  a  new  item  for  which  no  speci¬ 
fic  historical  data  are  available,  nor  are  directly  applicable  Cost  Estioa 
ting  Relationships  (CIS)  present. 

Olvera:  An  item  A  is  similar  to  item  B  in  terms  of  function  and  technical 
characteristics  except  that  new  technology  will  allow  for  replacement  of, 
say,  subassmnblles  G  and  H  by  X  and  Y.  The  unit  cost  of  item  A  ie  current 
ly  $1000. 

Procedure:  Development  personnel  "estimate"  that  subassembly  0  accounts 
for  lo*  ($100)  of  the  unit  price  and  H  accounts  for  15)6  ($150).  Without 
these  subassemblies,  item  A  would  therefore  cost  $750.  Now  the  engineer's 
cost  estimate  is  that  both  X  and  Y  will  cost  50)6  more  than  G  and  H  or  a 
total  of  $375*  This  added  to  the  cost  of  the  similarities  of  A  and  B 
($750)  yields  a  final  unit  cost  estimate  for  item  B  of  $1125* 

Although  the  above  example  might  appear  oversimplified,  the  baalc 
approaches  and  steps  are  readily  evident,  and  the  reader  should  be  able 
to  see  how  such  an  approach  is  used  in  greater  complexity.  Also,  one 
should  be  able  to  see  the  use  of  analogy  on  a  wider  basis  and  thus  see  the 
possible  application  to  estimating  development  time,  effort,  and  cost. 
However,  It  should  be  made  clear  that  the  above  example  depicts  the  proce¬ 
dure  for  a  poor  estimate  by  analogy,  that  there  exist  many  hidden  assump¬ 
tions,  and  the  rationale  of  the  estimate  is  far  from  sufficient  for  practi 
cal  use. 


B.  Hidden  Assumptions.  In  reviewing  the  above  example  carefully,  the 
following  assumptions  can  be  noted  even  though  they  are  not  specified: 

1.  Only  one  analogous  equipment  is  available. 

2.  Subassemblies  are  easily  defined. 

3-  Changes  in  subassemblies  do  not  present  any  required  modification 
in  the  remainder  of  the  item. 

fc.  TOie  apportionment  of  cost  to  subassembly  level  is  known. 

5.  Material,  labor,  and  process  differences  are  known  to  the  extent 
that  the  50)6  cost  increase  can  be  estimated. 

6.  The  cost-quantity  relationship  and  point  of  comparison  is  the  same 
for  each  item. 

7*  The  unit  prices  are  represent ive  of  equivalent  time  periods. 

These  "hidden  assumptions"  of  course  may  or  may  not  be  true,  but  the 
fact  that  they  can  be  listed,  strongly  points  out  the  fact  that  the 
example  was  very  deficient  in  back-up  rationale  for  the  estimate. 

C.  Lack  of  Adequate  Rationale.  Note  that  no  real  rationale  is  ex¬ 
pressed  in  the  above  example  and  many  questions  are  left  unanswered,  some 
of  which  follow.  There  is  no  explanation  why  item  A  was  selected  for  the 
analogy,  nor  why  only  item  A  was  selected.  Could  the  data  base  have  been 
enlarged  by  considering  other  existing  "items"  besides  item  A?  Even 
though  a  larger  data  base  of  additional  items  might  not  be  adequate  to 
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derive  a  CER,  they  might  be  used  for  analogy  purposes.  Drawing  the  analogy 
from  more  than  one  item  will  strengthen  the  confidence  of  the  final  estimate. 

Also,  when  considering  other  analogous  items,  the  need  that  they  be  of 
the  same  functional  family  does  not  necessarily  hold.  Basically,  certain  sub- 
assemblies  or  functional  portions  of  the  new  item  are  the  areas  where  the 
analogy  estimating  is  most  critical.  Do  such  subassemblies  exist  in  other 
items  of  different  functional  families,  and  if  so  were  these  considered? 

Biis  would  help  substantiate  the  rationale  for  the  cost  apportionment  esti¬ 
mate  for  the  subassemblies.  In  the  example,  no  rationale  was  given.  For 
on  estimator  to  say  a  portion  of  an  item  accounts  for  10  or  15  percent  of 
the  total  cost  requires  an  explanation  of  how  such  a  proportion  was  arrived 
at.  If  the  estimate  was  "expert  Judgment"  it  should  be  stated  as  such  and 
a  written  rationale  backing  up  the  judgment  should  be  present.  If  the  esti¬ 
mate  was  derived  ftrcm  spare  parts  procurements  (spare  part  coBt  vs  total 
item  cost)  was  any  check  made  on  the  remaining  "parts"  costs?  'Hie  total 
unit  cost  of  item  A  should  also  be  explained  in  terms  of  prototype  cost, 
production  cost  and  the  year  that  the  cost  had  occurred.  Also,  how  many 
of  the  items  were  procured?  Has  the  price  about  stabilized?  Is  it  a  com¬ 
petitive  price  or  sole  source?  Is  the  quantity  used  comparable  to  the 
quantity  of  the  pew  item  being  procured?  If  not,  what  procedures  or  logic 
were  used  for  the  estimate?  The  answers  to  these  questions  are  necessary 
to,  in  part,  also  substantiate  the  "50  percent"  cost  Increase  for  the  new 
subassemblies  over  the  existing  ones.  Other  rationale  is  further  required 
to  back  up  the  "50  percent",  such  as  consideration  of  material  costs,  labor 
costs,  and  maturity  of  manufacturing  process  and  market. 

A  final  area  to  consider  in  the  rationale  is  the  addressing  of  possible 
interaction  between  the  newly  incorporated  features  and  the  "similar"  por¬ 
tions.  Are  there  any  engineering  problems  created  by  the  substitution  of 
subassemblies?  Will  the  remaining  areas  remain  the  same  in  function,  but 
possibly  not  in  layout  or  configuration?  If  the  similar  portions  are 
altered,  the  production  cost  history  may  well  have  to  be  looked  at  care¬ 
fully  to  account  for  the  establishing  of  a  new  line  and  the  provision  for 
being  on  a  different  part  of  the  learning  curve. 

Without  a  written  statement  addressing  the  above  points  of  backup 
rationale,  the  user  or  recipient  of  the  estimate  would  have  difficulty 
accepting  the  estimate  with  any  degree  of  confidence.  Also,  even  the  esti¬ 
mator  at  a  later  date  would  have  difficulty  reproducing  the  estimate  al¬ 
though  knowing  what  the  final  figure  was  he  could  probably  back  into  it 
again  with  some  rationale,  probably  not  the  same  as  the  first  time.  There 
is  a  need  then,  to  emphasize  the  requirement  for  rationale  and  especially 
the  documentation  of  that  rationale.  Estimating  by  analogy  should  not  be 
considered  an  easy  method  and  should  not  be  used  in  a  slip-shod  manner  or 
presented  as  a  cursory  endeavor.  The  next  section  will  expand  further  on 
the  possibilities  and  especially  the  problems  of  using  the  analogy  method 
for  cost  estimating. 
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IV.  Problems  Inherent  In  Analogy  Estimating. 

A.  Relationships  to  Other  Methods.  In  either  an  implicit  or  explicit 
■anner,  all  cost  estimates  can  he  considered,  to  derive  from  an  analogy  base. 
The  statistically  derived  cost  estimating  relationship  (CER)  implies  that 
the  data  points  used  are  related  by  some  analogous  set  of  parameters,  per¬ 
formance  or  physical  characteristics,  and  for  this  reason  the  CIR  user  is 
always  cautioned  in  the  extension  and  use  of  the  CER.  In  the  CER,  the  sta¬ 
tistical  measures  of  "goodness  of  fit"  and  explained  variations  imply  the 
strengths  of  the  analogy.  Much  has  been  written  about  the  warnings  against 
using  an  "old"  CER  to  project  or  estimate  the  cost  of  items  outside  the 
limits  of  the  CER.  Usually  this  encompasses  extreme  extension  of  the  para¬ 
meters  and/or  a  major  difference  in  technology.  These  points  will  be  treated 
later  since  they  are  basically  drawing  an  analogy  on  a  previous  analogy,  and 
therefore  the  underlying  assumptions  will  have  to  be  investigated. 

Explicit  cost  estimating  by  analogy  occurs  when  the  estimator  uses  one 
or  more  similar  items  as  a  basis  for  developing  the  cost  estimate  for  a  new 
Item.  Tbls  Is  the  most  customary  use  if  the  analogy  and  Is  usually  without 
statistical  backing  although  the  possibility  of  using  statistical  methods 
might  be  applicable. 

Even  the  estimate  made  by  "expert"  Judgment  is,  in  a  way,  the  use  of 
analogies.  The  expert  draws  on  past  experience  and  logical  reasoning  to 
develop  his  estimate.  Although  specific  data  points  may  not  be  capable 
of  being  recorded,  the  general  points  considered  and  treated  by  the  expert 
can  be  put  down  on  paper  for  the  record.  In  fact,  the  exercise  of  writing 
such  a  rationale  could  well  provide  the  expert  with  the  desire  to  give  the 
estimate  more  critical  thought. 

B.  Uncertainty  (Technical.  Schedule,  Economic).  When  developing  a 
Life  Cycle  Cost  Estimate  (lCCEJ  by  analogy,  the  three  basic  areas  of  R&D, 
Investment  and  Operating  must  be  addressed.  The  estimator  usually  concen¬ 
trates  first  in  the  "unit  price"  or  that  price  to  be  paid  when  the  item  Is 
in  production.  The  second  area  is  usually  the  engineering  effort  required 
to  design,  develop,  and  bring  the  item  to  the  point  of  production.  The 
final  area  is  that  of  the  costs  of  operating  and  maintaining  the  item  after 
it  has  been  procured. 

It  is  the  interaction  of  these  three  areas  plus  possibly  undue  optimism 
or  project  enthusiasm  that  so  often  leads  the  way  to  poor  cost  forecasts  and 
estimates. 

In  reviewing  the  first  area,  unit  price,  one  may  consider  that  a  priori 
knowledge  exists  in  the  form  of  technical  characteristics,  weight,  size,  and 
cost  constraints.  However,  unknowns  exist  in  the  form  of  possible  technolog¬ 
ical  change  effects,  economic  change  effects,  and  schedule  change  effects. 
These  three  effects  cause  further  complication  through  their  interaction. 
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Contemplated  changes  in  technology  refer  to  a  certain  degree  of  risk  or 
uncertainty.  This  uncertainty  i6  then  further  affected  by  economic  uncer¬ 
tainty  which  might  be  specific  to  the  technology  or  more  general  in  nature. 
Technological  risk  further  implies  uncertainty  as  to  forecasting  schedules, 
thus  further  involving  economic  change  effects  for  different  time  frames. 

For  example,  if  an  estimator  is  dealing  with  the  problem  of  estimating 
a  new  radio  set,  it  is  reasonable  to  believe  that  the  technical  characteris¬ 
tics  for  the  most  part  will  be  similar  to  past  radio  sets.  However,  the 
incorporation  of  new  technology  may  be  the  main  purpose,  say  to  achieve 
smaller  size,  lesser  weight,  higher  reliability,  or  other  such  goals.  Ibis 
places  one  in  the  position  that  the  analogy  must  be  drawn  with  respect  to 
technological  change,  not  to  the  technical  or  performance  characteristics 
of  past  systems.  To  accomplish  this  requires  detailed  thought,  for  the 
premise  ttu.t  such  a  development  is  planned  implies  that  some  conclusion  has 
been  reached,  even  if  oubjective,  of  the  economic  efficacy  of  the  proposal. 

One  should,  however,  try  to  keep  this  assumption  out  of  mind  and  concen¬ 
trate  on  the  details  of  analogy.  When  dealing  with  the  areas  of  technical., 
economic,  and  schedule  uncertainty  in  terms  of  the  analogy,  the  effort  should 
be  concentrated  on  the  difference  of  design,  of  effort  required  to  redesign, 
of  parts,  module  and  labor  costs,  etc.,  at  present  prices  and  market,  simi¬ 
lar  costs  at  forecast  prices  and  market,  and  rationale  for  the  difference  in 
prices.  Component  development  is  of  an  almost  continuing  nature,  thereby  in 
effect  changing  the  overall  state  of  electronic  technology.  Early  use  of 
the  new  components  in  equipment  development  can  be  considered  the  very  mo¬ 
mentum  of  equipment  change.  However,  early  use  of  such  newly  developed  com¬ 
ponents  and  parts  finds  their  prices  high  because  of  the  lack  of  sufficient 
market  size  and  production  capabilities.  This  could  lead  to  either  estimates 
that  are  too  high  or  estimates  that  are  too  low  depending  upon  the  assump¬ 
tions  used  in  forecasting  the  future  market  environment.  Such  questions  as, 
"will  there  be  a  commercial  as  well  as  military  market  for  the  new  ccmpdnets?" 
should  be  considered.  Will  the  environmental  and  stress  requirements  unduly 
separate  the  two  market  segments?  What  patent  conditions  prevail  and  what 
similar  components  are  being  marketed?  What  about  market  timing?  Can  it 
be  expected  that  lowest  cost  will  occur  when  it  is  planned  to  produce  and 
procure  the  new  equipment  or  will  the  market  stabilize  sometime  after  first 
procurement  of  the  equipment? 

The  thought  comes  immediately  to  mind  that  the  engineer  or  developer  has 
enough  problems  in  the  technical  area  and  cannot  (and  probably  should  not) 
expend  the  time  and  effort  necessary  for  "market  research."  However,  in  hi6 
estimate  he  implies  market  knowledge.  An  alternative  that  might  be  consi¬ 
dered  is  to  select  some  person  or  persons  and  task  them  with  a  general  re¬ 
sponsibility  for  maintaining  at  least  gross  cost  trends  in  the  component 
field,  much  like  technological  trends  noted  by  technological  forecasters. 

In  this  way,  where  the  present  part  price  is  known  by  the  engineer,  the 
"market  monitor"  might  be  able  to  project  more  quantitatively  the  "production" 
price  for  a  given  time  frame. 
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People  like  to  beJieve  that  items,  the  cost  of  which  they  must  estimate, 
are  completely  new  and  unique,  'nils  is  especially  prevalent  in  military  pro¬ 
grams  since  the  commercial  world  doesn't  have  missiles,  tanks,  and  complex 
weapon  systems.  However,  upon  reflection  is  should  be  noted  that  although 
the  end  use  and  configuration  of  an  item  might  not  have  a  commercial  counter¬ 
part,  the  technology,  materials,  manufacturing  processes,  and  even  components 
do  exist  in  the  civilian  sector.  Scientific  and  technological  state-of-the- 
art  is  not  unique  to  a  given  investigator,  but  usually  permeates  the  entire 
scientific  community.  This  is  readily  apparent  when  a  breakthrough,  discovery, 
invention,  or  development  is  announced  by  an  individual  investigator.  In  very 
little  time,  publications  are  filled  with  the  findings  of  other  investigators 
who  were  pursuing  parallel  efforts. 

The  gap  in  time  between  discovery  and  the  development  of  the  discovery 
for  useful  application  is,  in  reality,  the  greatest  obstacle  that  besets  the 
estimator  and  sets  the  commercial  and  military  sectors  aparts.  The  commer¬ 
cial  world  enjoys  the  advantages  of  that  yardstick  called  the  "market"  where 
prices  tire  6et  in  terms  of  supply  and  demand.  Competitive  forces  tend  to 
restrain  the  Introduction  of  the  "new"  until  the  price  and  market  size  appear 
adequate  to  yield  a  profit. 

The  estimator  of  costs  for  military  items  does  not  find  himself  in  this 
same  supply  and  demand  market.  Instead,  he  must  evaluate  for  acceptable 
"prices"  in  a  strategic  and  tactical  competitive  "threat"  market  place, 
where  the  actual  competitive  pressure  is  to  maintain  a  superior  defense  pos¬ 
ture  in  an  uncertain  environment.  When  the  opportunity  to  develop  a  superior 
item  presents  itself,  the  estimator  must  estimate  the  development  time  and 
cost,  the  production  time  and  cost,  and  the  impact  on  field  costs.  Time  is 
a  transcending  variable  here  and  its  economic  effects  should  be  apparent . 
Failures  or  delays  during  development  will  result  in  stretch-out  coupled 
with  increased  coats.  Inflationary  effects  will  be  making  their  mark  through¬ 
out  the  program.  Component  and  material  markets  as  well  as  manufacturing 
processes  might  mature  significantly  with  time,  lowering  ultimate  item  costs. 
Early  goals  of  MTBFs  and  logistic  concepts  may  not  be  achieved  in  the  field 
and  these  costs  may  be  increased.  All  these  are  possible  areas  of  concern 
when  dealing  with  uncertainty  and  it  must  be  dealt  with  early  on  the  first 
estimate  when  the  fielding  target  date  is  six  years  or  so  downstream. 

Analogies  can  be  helpful  in  evaluating  uncertainty,  but  again  care  must  be 
taken  in  defining  the  assumptions  and  furnishing  the  rationale. 

V.  Procedures  to  Consider.  The  first  step  to  be  undertaken  in  cost  esti¬ 
mating  is  basic  to  all  approaches,  including  analogy;  determine  and  state 
comprehensively  what  is  to  be  estimated.  This  should  include  more  than 
naming  the  item  and  listing  its  desired  attributes.  Specify  the  likely  time 
requirements  and  proposed  quantities.  List  the  predecessor  items  if  the  item 
is  of  an  evolutionary  class.  Spell  out  what  makes  this  item  new.  Don't 
limit  this  to  performance  -  relate  it  to  parts,  circuits,  materials,  processes, 
etc.,  wherever  possible.  Now  the  analogy  process  can  begin. 
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Conduct  a  search  for  comparable  items.  In  doing  this  make  a  concerted 
effort  to  go  beyond  the  obvious.  Include  commercial  Items  when  possible. 
Subdivide  the  item  to  be  estimated  into  functional  and/or  component  cate¬ 
gories.  List  these  elements  in  the  order  of  their  estimated  contribution 
to  overall  cost.  Those  elements  that  are  thought  to  impact  the  greatest 
on  the  overall  cost  should  be  concentrated  upon.  Search  for  comparable 
elements.  At  the  element  level,  this  may  lead  outside  the  obvious  generic 
family  of  the  overall  item.  Parallel  the  search  for  comparable  elements 
with  a  search  for  CERs.  On  the  first  run  through  don't  evaluate  too  criti¬ 
cally  the  CERs  found.  If  they  appear  to  have  some  merit,  keep  them  for  later 
evaluation.  Remember  that  the  more  methods  or  approaches  that  can  be  under¬ 
taken  to  develop  the  estimate,  the  stronger  the  rationale  and  the  confidence 
will  be  for  that  estimate. 

Normally,  the  greatest  attention  should  be  applied  to  those  elements  of 
the  overall  item  that  are  thought  to  be  the  most  expensive.  Be  careful  here 
though,  because  the  less  costly  elements  may  veil,  collectively,  account  for 
the  greatest  portion  of  the  item.  This  can  be  especially  true  when  deploy¬ 
ment  and  numbers  are  considered.  Oftentimes,  a  lover  cost  element  when 
multiplied  by  the  number  to  be  deployed  can  result  in  an  overall  cost  greater 
than  the  overall  cost  of  a  higher  cost  element  vhlch  vlll  be  deployed  in 
lover  numbers.  Combinations  of  lover  cost  elements  can  pyramid. 

Nov,  for  the  major  item  to  be  cost  estimated  and  for  its  elements  there 
should  exist  a  "data  base"  of  sorts.  Note  that  no  actual  estimating  has  been 
recommended  up  to  this  point,  nor  has  a  real  evaluation  been  made  of  the 
"data".  Purposely,  this  has  been  done  to  keep  Tram  forming  a  mental  set  on 
a  figure  or  figures  that  might  provide  motivation  for  discarding  non-confirm¬ 
ing  data  prematurely. 

In  evaluating  the  data,  the  first  thing  to  look  at  are  any  CERs  that  vere 
found.  If  valid  CERs  are  found,  hardvare  cost  estimating  by  analogy  is  hot 
necessary.  However,  CERs  are  sometimes  misapplied;  therefore,  the  user  is 
cautioned  to  "know"  the  CER  before  relying  on  it  for  the  estimate.  It  is 
difficult  to  delineate  all  the  pitfalls  that  can  be  encountered  with  a  CER, 
but  some  of  the  more  obvious  evaluation  criteria  can  be  listed.  For  example, 
the  age  of  the  CER  could  make  its  use  questionable.  The  age  should  be  analy¬ 
zed  carefully,  and  this  can  only  be  done  effectively  if  the  supporting  data 
base  is  known.  A  review  of  the  data  base  should  include  a  determination  of 
differences  in  technology  as  veil  as  price  level  changes.  Thought  should  be 
given  to  possible  adjustment  of  the  CER  if  the  relationship  is  still  logical 
and  a  rationale  for  the  adjustment  is  evident.  Even  if  the  CER  cannot  be 
adequately  adjusted,  keep  it  for  the  time  being  and  later  it  might  be  used 
In  the  analogy  approach. 

Continuing  further  on  the  subject  of  CERs,  it  is  now  time  to  look  at  the 
analogous  items  and  elements  of  the  data  base  and  attempt  to  develop  CEPs 
from  them.  Document  the  findings. 
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Row  it  can  be  assumed  that  the  CER  approach  is  ended  and  what  is  left  is 
to  develop  the  necessary  analogies. 

The  major  item  analogy  should  be  taken  on  first.  Start  by  listing  the 
similarities  and  differences  with  the  analogous  items.  Establish  a  compar¬ 
able  market  price  situation  for  the  analogous  items.  That  is,  make  the 
necessary  adjustments  for  quantity  comparability  and  for  the  economic  effects 
of  time.  In  this  regard,  make  use  of  historical  improvement  curve  trends, 
when  possible,  and  the  necessary  Inflation/Escalation  Guidance  established 
by  the  command.  On  commercial  items  used  in  the  analogy,  quant lty/prlce 
relationships  may  not  be  available.  If  such  data  are  not  available,  make  a 
statement  to  that  effect  to  assure  that  comparability  adjustment  is  not 
assumed. 

A  table  of  data  should  now  exist  in  a  form  somewhat  like  the  following: 

Element/Function  New  Items  Analogous  Items 

_  1  2  .  .  .  m 


A 

B 


X 


Basic  Cost 

$X 

ro 

• 

•  •  K 

Adjusted  Cost 

— 

$Z1 

$z2  . 

•  ■ 

The  cells  of  the  table  compare  the  similarities/differences  of  the  vari¬ 
ous  items  in  whatever  tenns  these  elements  are  normally  expressed  in.  This 
table  should  result  in  highlighting  the  areas  of  major  dissimilarities  on 
which  further  estimating  effort  will  have  to  be  concentrated.  The  Importance 
of  the  search  for  analogous  sub -items  or  elements  should  now  become  apparent, 
for  this  is  where  the  estimating  procedure  has  arrived.  It  should  be  possi¬ 
ble  to  conduct  a  more  critical  and  quantitative  approach  to  the  estimate  now 
instead  of  the  "Judgment"  approach  previously  critiqued.  Judgment  must  still 
be  present,  however,  in  making  the  choices  of  what  data  to  use  and  how  to  use 
it.  Again,  it  is  highly  recoanended  that  such  decisions  be  documented. 
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The  attempt  here  Is  to  modify  Yi  in  some  logical  manner  to  arrive  at  $X. 

If  only  one  analogous  item  (Y^)  were  used, the  result  would  tend  to  be  a  point 
estimate  for  $X.  The  use  of  more  than  one  item  should,  however,  lead  to  a 
range  of  $X's,  thus  providing  a  better  display  of  uncertainty.  Also,  it 
should  provide  the  impetus  for  an  iterative  procedure  of  returning  to  modi¬ 
fiers  for  further  review.  The  modifiers  may  take  on  the  form 

$X  =  kY  +  J 

Where  k  represents  the  effect  of  differences  in  similar  elements  and  J 
represents  additional  elements  not  found  in  Y.  The  modifier  "j"  can  also  be 
seen  to  take  on  a  negative  sign  when  Y  contains  elements  that  do  not  exist  in 
X.  This  appears  to  be  a  restatement  of  the  original  analogy  that  was  criti¬ 
qued,  but  it  should  be  noted  that  contributing  percentages  are  not  used  in 
this  approach.  Attention  is  being  placed  on  the  "differences"  and  adjustment 
will  be  made  to  the  "total"  cost.  Gross  cost  estimates  of  these  differences 
based  on  complexity,  technology,  and  weight  may  well  be  better  than  detailed 
analysis  of  performance  and  functions.  For  example,  the  cost  to  produce  ele¬ 
ments  consisting  -of  discretee  components  should  be  more  a  function  of  com¬ 
plexity  and  weight  than  of  whether  the  element  is  a  preamplifier  or  a  modem. 

To  the  people  in  the  manufacturing  environment,  their  view  ie  one  of  fabrica¬ 
ting  and/or  assembling  chassis'  or  boards  containing  so  many  active  and  passive 
elements.  Their  concern  is  not  what  the  item  does  eventually.  Saying  this 
should  not  be  taken  to  belittle  the  complexity  and  expertise  of  any  commodity 
area.  That  expertise  manifests  itself  in  the  development,  design,  layout, 
and  packaging  of  an  item,  but  not  necessarily  in  production.  Thought  would 
have  to  be  given  here  to  any  special  deviations  from  normal  production  pro¬ 
cesses  or  to  any  unique  components. 

In  specifically  expressing  the  use  of  gross  estimating,  an  attempt  is  being 
made  to  narrow  down  the  Judgment  process,  and  to  stay  above  the  piece  part 
pricing  techniques.  Even  where  piece  part  data  are  available  to  some  extent 
and  the  parts  are  highly  repetitive  in  the  design,  care  must  be  taken.  Multi¬ 
plying  small  cost  figures  (e.g.,  4  per  weight)  be  large  numbers  can  result  in 
greater  errors  than  gross  estimating  by  complexity  analogy. 

If  the  area  of  difference  to  be  estimated  happens  to  be  6ome  discrete 
defineable  subassembly,  comparison  to  other  military  and  commercial  items  may 
be  more  evident.  A  commercially  based  CER  might  also  be  used  to  advantage. 

If  apalyzlng  other  military  and  commercial  items  makes  CER  derivation  appear 
impossible,  one  approach  would  be  to  plot  the  scatter  diagram  of  the  data 
available.  If  it  is  possible  to  group  the  data  similar  to  that  shown  in 
figure  1,  the  arithmetic  means  of  the  data  for  both  y  and  x  axes  may  be  obtained 
and  the  significance  between  the  means  can  be  tested  by  the  student's  t  test. 

It  is  evident  that  in  most  cases  the  significance  of  differences  in  the  y 
mean  values  will  present  no  great  difficulty  to  ascertain.  Much  more  effort 
should  be  placed  on  assuring  the  insignificant  difference  of  the  x  mean  values, 
•mis  should  be  done  on  all  the  major  contributing  parameters  (x  values)  of  the 
samples.  Once  this  is  accomplished  the  difference  in  y  mean  values  has  some 


9 


Performance 
Figure  1 

meaning.  This  will  proride  an  "average"  military  analogous  estimate  with  a 
range  denoting  the  uncertainty.  Also,  a  check  on  the  propriety  of  the  esti¬ 
mate  is  obtained  by  the  difference  of  the  means  which  in  essence,  is  a 
"militarization"  factor,  which  should  be  amenable  to  explanation.  If  this 
factor  appears  logical  and  can  be  supported  with  rationale,  e.g.,  low  pro¬ 
ductive  yield  on  parts  meeting  environmental  needs,  then  it  can  also  be  used 
to  sene  advantage  for  application  to  commercial  points  outside  the  grouped 
data. 

Another  method  occurs  when  a  CER  can  be  obtained  from  commercial  items. 
Often  such  a  CHI  is  possible,  but  military  counterparts  tend  to  group  about 
a  narrower  portion  of  the  CER  parameters.  An  example  of  this  1b  shown  in 
Figure  2. 


y 

$ 


x 

Performance 
Figure  2 


10 


Again,  the  method  of  comparing  means  can  be  used  and  In  addition,  If  the  para¬ 
meter  relationship  of  the  CER  is  deemed  to  be  logical  for  military  Items,  a 
parallel  CER  may  be  constructed  through  the  means  of  the  military  Items. 

Again,  the  "militarization"  factor  vlll  have  to  be  evaluated  for  feasibility. 

The  above  tvo  approaches,  as  outlined,  assume  rather  strongly  that  the 
basic  technology  and  complexity  are  comparable.  However,  very  similar 
approaches  can  be  taken  to  generate  factors  or  functions  of  technological 
change.  Figure  3  displays  vhat  might  happen,  in  general,  for  changes  in 
technology. 


Weight 
Figure  3 


As  indicated,  for  any  given  weight,  the  cost  might  be  found  to  Increase 
depending  upon  whether  one  is  considering  discrete  component  design  (d),  inte¬ 
grated  circuits  (IC),  or  medium,  large  scale* integration  (MSI/LSI).  Such  a 
relationship  would  probably  hold,  when  the  different  technologies  are  at 
different  periods  of  maturity.  As  the  newer  technologies  mature,  the  cost 
functions  should  tend  toward  the  lower  bounds. 


The  higher  cost  per  unit  of  weight  of  a  new  technology  should  not  be 
viewed  as  detrimental,  however,  since  as  shown  in  figure  4,  performance 


Figure  4 
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for  a  given  weight  will  be  expected  to  Increase  significantly  for  increases 
in  technology.  The  seme  methods  of  comparing  military  items  to  commercial 
items  treated  above  can  be  used  to  generate  or  estimate  the  effects  of  a 
change  in  technology.  Here,  it  is  assumed  that  the  more  mature  technology 
will  furnish  a  substantial  enough  base  to  allow  for  comparison  to  the  fever 
known  points  of  the  new  technologies.  Also,  as  shown  in  figure  5,  for  sane 
hypothetical  CER  ($  vs  parameters),  a  technology  function  may  be  derived. 

In  the  example  shown,  the  increase  in  performance  is  not  offset  by  a  decrease 
in  weight  in  terms  of  cost,  hence  an  increasing  f  (Tech). 


Figure  5 

It  should  be  realized  by  the  estimator  that  f  (Tech)  could  follow  the  basic 
CER  function,  thus  moving  along  the  performance  axis  for  increasing  technology. 
This  is  analogous  to  figure  U.  Also,  where  the  increase  in  performance  exceeds 
the  decrease  in  weight,  the  technology  function  may  take  on  negative  slope 
parameters,  thus  indicating  new  data  points  below  the  basic  CER. 

The  above  treatise  does  not  imply  that  the  estimator  must  derive  such 
functions,  but  is  offerred  to  aid  in  the  development  of  rationale.  If  a 
single  point  or  a  few  points  representing  never  technological  application  are 
available,  their  position  relative  to  an  older,  mature  technology  will  provide 
the  basis  for  estimating  higher  or  lower  by  use  of  analogy. 

Using  one  or  more  of  the  above  approaches  in  estimating  the  "diiferences" 
in  analogous  items  should  now  allow  for  adjustment  of  the  basic  item  price  to 
yield  an  estimate  for  the  new  item.  This  should  suffice  for  developing  the 
cost  estimate  of  the  item  and  now  permit  the  treatment  of  developments  costs 
and  operating  costs  by  analogy.  These  areas  are  more  tenuous  and  require 
greater  care  in  the  identification  and  treatment  of  assumptions.  Whereas 
the  estimation  of  a  "unit  price"  relied  heavily  on  historical  evidence,  more 
care  should  be  used  in  evaluating  past  efforts  in  development  or  operating 
costs.  Manufacturing  is  faced  with  quantity  output  to  a  demand/price  environment. 
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Item  performance  is  a  measureable  entity  and  in  order  to  compete,  a  manufac¬ 
turer  oust  consider  labor/capital  trade-offs  to  remain  competitive  in  price 
for  given  performance.  Development,  however,  is  faced  with  the  Judgment  or 
evaluation  of  the  best  technical  approach  proferred  and  the  validity  of  the 
price  or  value  is  not  easily  determinable.  Relying  rigidly  upon  historical 
development  times  and  cost  of  analogous  programs  can  be  seriously  misleading 
in  several  ways.  Two  axioms  are  normally  present  and  when  viewed  critically 
lend  some  insight  for  estimating.  One  is  that  "history  repeats  itself"  and 
the  other  is  that  "time  is  money".  The  repetition  of  history  is  the  most 
common  base  for  the  analogy  approach  of  making  the  estimate.  However,  it 
should  be  considered  that  estimates  become  goals,  goals  bring  on  controls, 
and  controls  are  usually  broken,  resulting  in  excess  cost.  So,  if  the 
estimate  is  based  on  history,  it  is  likely  to  fall  in  a  region  somewhat 
lower  than  the  cost  finally  realized.  In  essence,  the  estimates  tend  to 
predetermine  the  outcome  and  prolong  the  historical  chain,  thus  giving  a 
basis  for  "rule-of-thumb".  Such  a  chain  will  continue  to  reinforce  the 
rule -o  f -thumb . 

The  development  effort  should  be  analysed  carefully.  In  analogy  this 
must  be  done  against  a  historical  base.  It  can,  however,  be  done  on  a  seg¬ 
mented  basis,  such  as  the  effort  to  derive  the  design  plan,  the  amount  and 
type  of  testing  required,  the  building  of  prototypes,  etc.  The  time  and 
effort  analogy  should  be  based  on  the  scope  of  a  program,  -e.g.,  a  completely 
new  design  of  new  materials,  a  new  design  of  present  technology,  a  redesign 
of  equipment,  an  upgrading,  etc.  Consideration  should  be  given  to  the  "state- 
cf -knowledge"  present  at  the  time  of  the  estimate  -  how  far  has  prior  develop¬ 
ment  been  carried,  e.g.,  through  AD  models,  etc?  Also,  remember  that  time  is 
money  and  development  procurements  can  be  likened  to  an  employment  game,  since 
a  defense  contractor's  major  assets  are  his  knowledge  and  skills  formation 
(people).  It  is  not  unheard  of  for  defense  oriented  employers  to  realize  a 
return  on  capital  investment  extremely  higher  than  the  actual  realized  profit. 
The  name  of  the  game  for  a  contractor  is  employment,  and  his  goals  are  prbfit- 
able  continuity  and  expansion.  The  Government ' s  goals  are  adequate  item 
performance,  minimization  of  costs,  and  shortening  of  development  time.  The 
goals  of  the  two  parties  conflict  as  in  any  game  and  it  might  be  said  that 
the  Government  can  lose  more  than  the  contractor  can  gain.  For  the  purpose 
of  this  paper,  the  main  point  is  that  the  strategy  must  be  to  counteract  the 
potential  contractor's  goals  to  a  point  satisfying  both  parties.  To  accomplish 
this,  the  estimator  and  manager  should  consider  whether  or  not  analogous  pro¬ 
grams  could  have  been  accomplished  in  shorter  time  and  if  actors  that  caused 
time  delays  previously  can  be  overcome.  These  are  points  that  should  be 
addressed  specifically  in  the  analogy  and  fully  documented. 

Operating  costs  present  the  most  difficult  portion  of  all  costs  either 
to  estimate  or  to  determine.  Because  there  is  a  paucity  of  data  representing 
actual  field  use,  it  is  difficult  to  structure  a  meaningful  analogy  on  an 
actual  use  basis.  For  this  reason,  most  operating  costs  are  estimated 
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deteralnistically  on  the  basis  of  estimated  or  factory  tested  MTBF  and  MFTR 
along  vlth  various  factors  covering  the  several  cost  elements  of  the  operating 
phase.  Basically,  this  methodology  is  not  incorrect,  but  one  might  normally 
expect  that  It  underestimates  the  costs  because  it  assumes  that  non-field  data 
is  fully  representative  of  field  data.  Sensitivity  analysis  is  usually  intro¬ 
duced  to  demonstrate  the  effects  of  the  uncertainty,  but  does  not  help  in 
determining  the  most  likely  costs.  Analogy  may  be  helpful  here,  if  some  field 
data  sire  available.  It  has  been  said  that  MTBF  decreases  as  the  distance  from 
the  manufacturer  increases.  How  true  this  is  must  be  considered,  for  not  only 
would  it  affect  the  cost  estimates  but  it  would  affect  the  operating  philosphy 
as  well.  In  sin  attempt  to  account  for  these  changes,  PEMA  secondary  item 
requisition  for  non-common  parts  of  analogous  equipments  might  provide  an  idea 
of  whether  or  not  consumption  was  greater  or  less  than  expected.  Also,  on 
analogous  items,  reports  of  field  problems  should  be  ferretted  out  where 
possible.  The  new  concept  of  Operational  Testing  by  "users"  and  the  use  of 
elapsed  time  indicators  should  provide  better  relationships  in  the  future  for 
plant  vs  field  test  results. 

Each  cost  factor  or  computational  algorithm  should  be  scrutinized  care¬ 
fully  in  terms  of  basic  assumptions  and  possible  magnitude  of  error  generation. 
All  operating  costs  should  be  looked  at  in  terms  of  constant  dollars  and  in¬ 
flated  dollars  and  consideration  should  be  given  to  program  slippage. 

VI.  CO!fC  HIDING  REMARKS.  Cost  estimation  by  analogy  is  probably  one  of  the 
most  powerful  techniques  for  estimating  if  a  dedicated  effort  is  pursued. 
However,  history  has  shown  it  to  be  severely  lacking  in  credibility  because 
it  is  too  easy  to  do  a  slip-shod  Job  to  meet  "rush"  requirements.  If  analogy 
is  to  gain  the  proper  respect  due  it,  its  future  use  must  reflect  comprehen¬ 
sive,  well  documented  efforts  backed  by  very  Btrong  rationale.  The  develop¬ 
ment  of  Cost  Estimating  Relationships  (CER)  has  become  the  major  thrust  of 
many,  but  when  viewed  in  their  proper  perspective,  CTOs  are  also  analogies. 

Very  seldom  is  it  possible  to  find  or  develop  CERs  for  all  that  needs  to  be 
estimated.  Also,  CERs  can  be  hazardous  in  blind  application  even  if  they 
are  accompanied  by  good  "statistics".  In  the  final  analysis,  analogy  trans¬ 
cends  all  "parametric"  estimating  and  finds  use  in  all  phases  of  the  develop¬ 
ment-production-operation  cycle.  The  foregoing  section  of  this  paper  has 
delineated  many  approaches  that  can  be  taken  in  estimating  by  analogy.  The 
treatment  is  not  exhaustive,  but  hopefully  is  motivating  in  nature  toward 
better  use  of  analogy. 

The  following  is  a  summary  check  list  of  questions  the  estimator  might 
ask  himself  to  determine  how  comprehensively  he  has  conducted  his  effort  of 
estimating  by  analogy. 
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Check  List: 


For  Item  Cost,  Have  I: 

1.  Comprehensively  defined  the  item  to  be  estimated? 

2.  Broken  the  item  down  by:  function? 

components? 

3.  Ranked  the  relative  breakouts  by  estimated  co6t  importance? 

4.  Conducted  a  thorough  search  for:  comparable  military  items? 

comparable  ccmmerclal  items? 
comparable  contributing  elements? 

5.  Conducted  a  thorough  search  for  CERs? 

6.  Assured  that  the  "data  base"  established  from  the  search  contains 
all  pertinent  information  such  as:  dates? 

quantities? 

characteristics? 

.  technology  types? 

problems? 

7.  Identified  and  documented  the  data  sources? 

8.  Evaluated  the  data  and  completely  documented  the  evaluation? 

9.  Adjusted  or  normalized  the  data  for  use? 

10.  Documented  the  adjustments  and  their  reasons? 

11.  Set  up  a  table  of  data  (element /function  vs  items)? 

12.  Isolated  the  "differences"  between  the  item  to  be  estimated  and  the 
analogous  items? 

13.  Used  CERs  to  the  extent  possible? 

14.  Used  as  many  analogous  approaches  as  practicable? 

15.  Adjusted  the  "total"  item  cost  to  account  for  effects  of  the 
"differences"? 

16.  Documented  the  entire  effort? 

For  Development  Costs,  Have  I: 

1.  Defined  the  type  effort  (new,  redesign,  etc.)? 

2.  Conducted  a  search  of  similar  past  efforts? 

3.  Conducted  a  search  of  similar  sub-efforts? 

design  plan? 

testing? 

etc? 

4.  Reviewed  problems  encountered  on  past  efforts? 

5.  Documented  and  validated  any  "rules-of-thumb"? 

6.  Drawn  as  many  time  and  effort  analogies  as  practicable? 

7.  Documented  the  entire  effort? 
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For  Operating  Costs,  Have  I: 


1.  Made  a  deterministic  estimate  based  on  reliability  projections? 

£.  Conducted  sensitivity  analysis  on  the  key  variables? 

3*  Checked  analogous  equipments  to  determine  vhat  "actual"  vs 
"projected"  reliability  was? 

4.  Checked  for  extraordinary  FEMA  secondary  item  requisitions  on 
analogous  items? 

5.  Ferreted  out  any  reports  of  field  problems  on  analogous  items? 

6.  Incorporated  the  findings  of  4  and  5  above  in  the  substantiating 
of  the  sensitivity  analysis  (2)? 

7*  Documented  the  entire  effort? 


